Hapticon visual substitution system

ABSTRACT

A multiple anode photomultiplying image tube is provided employing an image intensifier structure and an array of anodes for converting visible information into patterned electrical signals. A modulating source associated with the tube assures that the electrical signals exhibit particular characteristics; and such signals are conveyed to an array of electrodes in contact with the skin of a subject, to be perceived by the subject as a conceptual image. The photomultiplying image tube can be mounted in or on spectacles, or on the head of the subject. Arrangements are disclosed for providing black and white information, color information, and stereoscopic information to the subject, and for using the eyes to aim the system at any desired object.

United States Patent 0 [72] Inventors Carter C. Collins;

Frank A. Saunders, Mill Valley; Julius M. Madey, San Anselmo, Calif.

[21 Appl. No. 804,550

[22] Filed Mar. 5, 1969 [45] Patented Feb. 9, 1971 [73] Assignee TheInstitute of Medical Sciences San Francisco, Calif.

[54] HAPTICON VISUAL SUBSTITUTION SYSTEM 22 Claims, 14 Drawing Figs.

52 u.s.c1 178/52, 35/35; l78/6.8, 178/72; 313/95, 313/98 511 muH04n7/18, 1104119/02 501 FieldotSearch 35 351;

178/6 lND, 6, 6.8, 5.2, 5.4; 313/95, 98

[56] References Cited UNITED STATES PATENTS 2,327,222 8/1943 Sell 35/1OBJECT OTHER REFERENCES Science 9/4/53 pp 277- 278 Perception by theSkin of Electrically Induced Vibrations" Vol. 118

Electronics 7/10/67 pp 44- 45 Primary Examiner-Robert L. RichardsonAttorney-Hall, Pollock and Vande Sande ABSTRACT: A multiple anodephotomultiplying image tube is provided employing an image intensifierstructure and an array of anodes for converting visible information intopatterned electrical signals. A modulating source associated with thetube assures that the electrical signals exhibit particularcharacteristics; and such signals are conveyed to an array of electrodesin contact with the skin of a subject, to be perceived by the subject asa conceptual image. The photomultiplying image tube can be mounted in oron spectacles, or on the head of the subject. Arrangements are disclosedfor providing black and white information, color information, andstereoscopic information to the subject, and for using the eyes to aimthe system at any desired object.

PATENTEDFEB 912m 3562.408

' sum 1 [1F 5 L/ OBJECT l t -y PULSE -H 20,us

| I CARRIER I (400 Hz) I PULSE 1 TRAIN I INVI'ZNTORS c. c. COLLINS F. A.SAUNDERS J.M.MADEY BY 7/ 4] @11 1/40,! Jami.

PATENTEUFEB 9mm 3,562,408

SHEET 2 OF 5 v THRESHOLD OF PAIN INDIVIDUAL v R|AT|ON U E l5 2 gPREFERRED ENVELOPE g OF OPERATION U m IO- D l D E O m THRESHOLD 0FFEELING l( ),us ICC us EFFECTIVE PULSE DURATION= pWper TRAIN I N I 20a.1 I

lNVI-JNTURS C. C. COLLINS F. A. SAUNDERS J.M.MADEY BY W d/M04 JULPATENTEDFEB slsn I I 3,562,408 sum 3 or 5 i 1 r P L T I I I 33c 7%%35oF/G. 3A

50 50o f O HAPTICON I 53 TUBE I 5| v 5|0 HAPTICON 5 O U TUBE i7 TO SKIN-HAPTICON TUBE FIG. 5A

INVENTORS C C COLLINS F. A. SAUNDERS J. M. MADEY "7 44 awk; V404, ,J

ATTORNEYS PATENTEUFEB 9:971 Q 3,562,408

SHEET 0F 5 INVENTORS c. c. COLLINS F. A. SAUNDERS J. MMADEY [HZ a1),I241 la b find...

ATTORNEYS PATENIEU FEB 9 I971 3 552 40 SHEET 5 BF. 5

R STIM.

6|- r HAPTICON 69 DRIVE R/L STIM.

f f L STIM. 67 690 70 INVENTORS C. C. COLLINS .F. A. SAUNDERS JMMADEYATTOR N EYS IIAPTICON VISUAL SUBSTITUTION SYSTEM BACKGROUND OF THEINVENTION Blindness has always been one of the most diffieult of allhuman afflictions to treat. Almost all therapeutic approaches to datehave been designed to rehabilitate-a blind person to a nonvisual world.However the relatively recent availability of sophisticated electronicequipment has'suggested to various workers in the field that it may bepossible to replace a lost sense of vision by conveying signals to thebrain in some alternative fashion.

In searching for an alternative modality to replace vision it has beenfound that the integ ument ofa-subject is the only organ besides the eyewith receptorsorganized to receive stimuli to be interpreted anddistributed in two dimensions of space, with temporal integratingcapability. It is richly innervated, and offers an excellent sensoryreceptor area for visual substitution system. In prior arrangements,artificial receptors such as TV cameras have delivered images to theskin by means of an array of tactile stimulators. It has been foundthat, with training, a blind subject can interpret the tactilerepresentation of an optical image picked up by the television camera,and thus essentially see through his skin. The image on the skin sets uppatterns of nerve pulses in the cutaneous nerves which arrive at thebrain as coded pulse patterns carrying optical information. The brainthen decodes this information based on experience acquired in training.Thus, a healthy intact sensory modality (cutaneous sensation) becomes ameans for carrying visual" infonnation from an artificial receptor suchas a television camera. l 7

Various problems have arisen with 4 systems suggested heretofore. Visualsubstitution systems employing a television camera associated with atactor arraymechanically vibrating on the subject's skin tend to berather large, bulky, and heavy, thereby limiting or actually eliminatingmobility of a person utilizing the visual substitution systemQThsesystem limitations are aggravated by the need for relatively highcapacity power sources, and associated cooling arrangements, necessaryfor proper operation of a mechanically vibrating arrangement of taetors.

In an effort to reduce somewhat thew eig ht of a visual substitutionsystem, it has been suggested. that direct electrical stimulation of theskin can be used in place of vibrating tac tors. Even with thismodification, however, the type of artifi cial receptor which must be.used (e.g., a TV camera) still makes the overall system relativelybulky, and of limited mobility. No truly practical alternative has beenavailable heretofore for a TV camera type of receptor. Any possiblealternatives which haven been suggested in prior patents or literaturehave, in practice, been found incapableof producing electrical stimuliof sufiicient strength and/or definition to achieve realistic conceptualimages. I

In. addition, earlier suggestions for electrical stimulation systemshave been subject to operational and utilizational 2 SUMMARY OF THEINVENTION In accordance with the present invention. a visualsubstitution system is provided employing a novel photomultiplier tubeincorporating an extended area photocathode an array of collectoranodes, and an image intensifier structure therebetween, for example: amicrochannel plate, "venetian blind" structure or other device. Thistube, which is termed herein a hapticon, is sufficiently small and lightin weight to permit its mounting directly on a pair of spectacle frames.or in a relatively small receptor unit carried directly on the head of asubject. The hapticon tube is associated with a pulse train generator ormodulating source adapted to convert an optical image, impressed on thephotocathode and intensified by the tubes image intensifier structure,into pulses at the tubes anode array which are then coupled viaappropriate conductors to an associated array of skin electrodesinengagement with the subject.

Attention is given to the characteristics of the pulses employed forelectrical stimulation, to assure that the stimulation achievessignificant but reliably pain-free stimulation of the subjects skin.More particularly the pulses are individually of rectangularconfiguration between I and 500 microseconds wide. Such pulses are timedto occur at repetition rates up to 10,000 pulses per second. The offtime between individual pulses permits at least partial recovery of theskin, repolarizution of the nerve cell membrane, and/or elimination ofmetabolic by products. To enhance these recovery effects, the pulsesare, in a preferred embodiment of the invention. applied in bursts of anumber of short pulses, with relatively longer time intervals betweeneach burst; and, in a preferred embodiment, the burst repetition ratemay range between 25 and I00 pulse bursts per second. By observing theseconditions, fading, habituation, adaptation, and other phenomenadescriptive of changing sensation levels are avoided, and gross skinchanges are eliminated even though the stimulation may occur constantlyfor a number of hours.

In its preferred form, the potential of the skin stimulating pulse iselectronically controlled so as to produce constant current pulses inthe skin having a magnitude sufficient to achieve meaningful butpain-free stimulation; and a manual control is incorporated in theequipment to permit adjustment of these parameters by the subjectthereby to accommodate various skin impedances and intensitypreferences. Such control means, and the waveform generator itself, areincorporated into physically small and lightweight circuit arrangementscapable of being mounted, for example, within the temples of a pair ofspectacles. As a result, the overall receptor, electronic control, andmanual adjustment facility represent compact, lightweight componentswhich can be readily wom on the head of a subject, or hand held, withsubstantially no physical discomfort.

In preferred forms of the invention, zoom optics may be incorporatedinto the receptor to permit the subject, at will, to examine an objectin varying degrees of detail. Other features of the system includeprovision for color interpretation through the use of one or morehapticon tubes of the type described, associated with three primaryfilters and with three separate and unique carrier or modulatingfrequency sources conveying information as to each primary color to thesubject. Stereoscopic presentation is also achieved through the use ofoptical arrangements adapted to shift periodically the subjects point ofview, in cooperation with a separate stimulating problems of visualsubstitution systemssuggested heretofore,

system conveying information to the subject as to the changing viewpoint. Still other arrangements are provided wherein ocular feedback isachieved through the use of contact mirrors wom by the blind eyes of thesubject and making use of the subject's normal eye movements to directthe angle of view of the hapticon camera receptor.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. I is a diagrammatic illustrationof a hapticon system constructed in accordance with the presentinvention;

FIGS. 2A through 2C inclusive illustrate some of the operatingcharacteristics of the FIG. I system;

FIG. 3 is an illustrative side view of a pair of spectaclesincorporating the hapticon system of FIG. I;

- FIG. 3A is a diagram of one form of zoom optics which may be employedin conjunction with the hapticon of the present invention;

FIG. 4 illustrates an alternative embodiment of the present invention;

FIG. 4A is an enlarged view ofthc hapticon assembly shown in FIG. 4;

FIG. 5A, 5B, and 5C respectively illustrate three different hapticonsystems constructed in accordance with the present invention adapted toconvey color information;

FIG. 6 is an illustration of a hapticon system constructed in accordancewith the present invention adapted to convey stereoscopic information;

FIG. 7 illustrates another embodiment of the present invention employingocular feedback; and

FIG. 7A is an illustrative view of a contact mirror arrangement whichmay be employed in the system of FIG. 7.

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 illustrates a hapticonvisual substitution system of the general type employed in the variousembodiments of the presentinvention. The system comprises aphotomultiplier image tube I0 of the particular construction to bedescribed more fully hereinafter, and termed a hapticon. Tube 10 isassociated with an optical system diagrammatically illustrated at I] forimaging an object 12 on the face oftube l0; and output signals from thetube I0 are conveyed by a conductor bundle [3 to a matrix of skinstimulating electrodes in contact with the skin 14 of a subject. Theelectrode matrix in contact with the skin 14 may take the form describedin the copending US application of C. C. Collins et al., Ser. No.800,948 filed Feb. 20, I969 for Flexible Cutaneous Interface."

Tube 10 comprises an evacuated envelope [5 enclosing a distributedphotoeathode 16, an anode array 17 of collector electrodes, and an imageintensifier structure positioned between elements I6 and I7 andconsisting of a secondary emission electron multiplier structure such asa microchannel plate 18 exhibiting an electron gain of between 10,000and 10,000,000. Microchannel plates of the type comprising the imageintensifier structure are in themselves known and are presently beingmanufactured by Zenith Radio Corp., Bendix Corp. Varian Associates, andothers.

Microchannel plate 18 comprises a glass disc, resembling a thin fiberoptics plate, having cored tunnels or holes etched therethrough. Thedisc or plate 18 may be the size of a small coin only I to 3 mm thick;and a 1 inch diameter plate can be provided with more than a millionholes exhibiting an interior surface area of more than 850 squarecentimeters. The inside of each of the etched-out glass tubes formed inplate 18 is coated with a few Angstroms of dielectric oxide. This highresistance coating, which acts as a source of secondary electrons, alsoserves to keep the electrons moving down each etched-out tube by varyingthe electric field across the ends of the tube. Metal contactsevaporated over the opposing faces of the disc or plate [8 allowpotentials to be applied to the ends of the tubes in the microchannelplate; and these differing potentials have been illustrated in FIG. I asV and V,. Photocathode 16 is maintained at a potential of substantiallyV and the collector anode electrodes 17 are at substantially groundpotential. The absolute magnitude of V is greater than that of V,, whichis in turn greater than that of V,.

In the-operation of the system shown in FIG. 1, anddisregarding for themoment the function of pulse train generator 20 (to be described), animage of object 12 focused by optical system ll on photocathode 16causes photoelectrons to be emitted from photocathode 16. Thesephotoelectrons are acceleratedtoward the rear of the tube by anacceleration potential (V,,) (V ).existing between photocathode I6 andthe front face of microchannel plate 18. These accelerated electronsenter the front ends of the several tubes formed in plate I8, anddislodge secondary electrons from the walls of said tubes. A fieldgradient voltage -I. (V,) creates a longitudinal electric field thatpulls these electrons down the tubes. producing even more secondaryelectrons from the tube walls. Plate I8 thus acts as a multiplyingstructure consisting of collimating tubes (although other similarstructures such as multiple venetian blind surfaces or the like could beused); and the repeated secondary emission occurring within each tubeproduces gains in the order of l0 without appreciable lateral diffusionof the amplified electron image.

The multiplied electron image continues to be accelerated by a uniformacceleration potcntial between the downstream face of plate I8 and thecollector anode array I7, and the resultant electron image beam currentis then intercepted by the various anodes constituting array I7. Theseveral anodes in array I7 are individually coupled to correspondingconductors in the conductor bundle or cable 13, and electricalstimulation signals are thus conveyed by the said conductors to suitableskin electrodes in contact with the subject's skin I4 to effect imagepatterned electrical stimulation of the skin for the purposes describedpreviously.

In order to avoid irritation with long term use, no DC polarizingcurrent should flow in the skin electrodes. Therefore, eithertransformer or capacitor coupling should be employed between each anodein array I7 and its respective skin electrode. to block DCcurrenLCoupling arrangements of this type can be elTectcd by appropriateprinted and/or integrated circuit techniques. or by discrete transformeror capacitor components ifsizc, cost, etc. permits.

In a preferred embodiment of the present. invention microchannel plateI8 is provided with, for example, 10.000 holes cooperating individuallywith, for example. L000 anodes in array 17; and in such an arrangement.each anode collects electron image beam current for ten multipliertubes. The overall tube 15 can be conveniently fabricated to exhibit anexternal diameter of I A0 2 inches...and a length of between I cm. and Iinch. The photocathode itself may take the form of a photoemissivedeposit on the inner front face of envelope l5, and can, for thedimensions specified, exhibit an active diameter of substantially Iinch. The transparent front face plate can be substantially 2 mm thick.Plate 18 can be substantially 3 mm thick, and the rear (anode) faceplatecan be substantially 2 mm thick. The remaining lengthdimensions betweenthe elements I6, 18 and 17 provide for adequate spacing between theseveral elements to avoid possible arcing; and the actual dimensionschosen for such spacing are dependent upon the magnitudes of the severalpotentials applied to the various elements ofthe hapticon tube 10.

It will be appreciated that the size of hapticon tube I0 is sufficientlysmall to to allow ready mounting of the tube in a spectacle frame toreplace the glass lenses which such a frame might otherwise carry. Insuch a location, an efficient, lightweight, and small optical imagingdevice of short focal length may be readily mounted in front of thehapticon photocathode surface to image an object on photocathode surface16. Considerations of this type will be discussed subsequently inreference to FIGS. 3 and 4.

The skin electrodes in contact with skin 14 are relatively small indiameter, and each electrode is preferably of coaxial configurationcomprising a central stimulating electrode having a diameter in theorder of one-eight inch surrounded by a coaxial ground electrode havinga diameter in the order of three-sixteenth inch. Other configurationscan be used such as random, linear or hexagonal arrangement ofnonconcentric, noncoaxial, or monopolar electrodes. The center to centerelectrode spacing between the several electrodes may be in the order ofone-fourth to one-half inch when applied to the skin of the back. butcan be of lesser size and spacing, e.g.,

one-eight inch center to center if they are to be applied to thediameter coaxial electrodes of these types limit current spread to theimmediate surface of the skin, thus tending to restrict stimulation totouch rather than to deeper (dull) pain receptors; and, when energizedin the manner to be described, achieve painless stimulation which hasbeen described variously as vibration, buzz," touch," or tickle.

In order to assure that the beam current intercepted by the anodes inarray 17, and coupled to the subjects skin 14, provides meaningful, butreliably pain-free electrical stimulation. a pulse train generator 20 isprovided to pulse and otherwise control the potential between elements16 and 18, or across element 18, or between elements 17 and 18, therebyto cause the electrical stimulation to occur as pulses exhibitingpreselected characteristics. Anodal pulse trains having a frequency of60 Hz., employing pulses having a width of l to 500 microseconds, havebeen employed to-produce skin currents of 2 to 20 milliamperes peak; thecurrent selected depending on pulse width. Adequate stimuli have beenfound to be as small as 50 microwatts, and the subjective level ofsensation can be accurately predicted from the current, pulse width, andnumber of pulses in a train. The sensation of continuously appliedpulses or pulse trains rapidly falls from an initial subjective level inless than one-half second. The electrical resistance of the back islower than that of the handsor arms. A stimulating current fall time of2 microseconds is observed due to the shunt capacity of the skinencountered by the electrode. The equivalent circuit of the skin of theback appears as 40,000 ohms in parallel with l0 picofarads. Anelectrolytic conducting electrode paste or lotion can reduce this tol0,000 ohms for more constant and reliable skin stimulation.

To achieve proper electrical stimulation, the pulse train generator 20is provided to control the characteristics of individual pulses appliedto the skin, the characteristics of a burst of such pulses applied tothe skin, and, optionally, the characteristics of a train of such pulsebursts. Unit 20 is illustrated in FIG. 1 as being connected betweenphotocathode l6 and plate 18; but it may, alternatively, be connectedacross plate 18, or between plate 18 and anodes l7. Pulses are generatedin unit 20, e.g., having the waveform shown in FIG. 2A. Each individualpulse in the pulse train should be between 1 and 5,00 microseconds inwidth, and should exhibit a rectangular waveform. FIG. 2A contemplates,for purposes of illus tration only, pulses having widths of 20microseconds. Pulse widths in the range of to I00 microseconds have beenfound to be comfortable, nonhabituating, easily recognizable as lightvibration, and quite different fromthe usualsensation as sociated withelectric shock. The sharp leading edge of each pulse is believed toachieve capacitive piercing of the external skin layer of deadepithelial cells, oil, etc., without the need for abrading the cells, ormechanical puncturing. The most useful pulse repetition rates of thecarrier include frequencies up to 10,000 Hz.; in other words, the timebetween pulses can be as low as 100 microseconds. The optimum rate(depicted in FIG. 2A) has been found to be about 400 Hz., and yields aperiod of 2.5 milliseconds. If each individual pulse is, for example, 20microseconds wide, 2,480 microseconds of the period is off-time," and isavailable for recovery, repolarization of the nerve cell membrane,and/or elimination of metabolic byproducts.

While a single pulse is capable of eliciting a sensation, it has beenfound that stimulation is more reliably felt when a burst of such pulsesis applied. The sensation thus is temporarily integrated physiologicallyin that bursts of e.g. 8 pulses are perceived as stronger stimulationthan bursts of e.g. 4 pulses, all other factors being equal. Such pulsebursts are effected by unit 20, which includes means for modulating theindividual pulse carrier by an on-off" switching action of the typeshown in FIG. 2A. If a 4 pulse burst is desired, the carrier of FIG. 2Ais turned on for 10 milliseconds (or 2.5 milliseconds times 4). A singlesuch interval is distinctly perceived. lf repetitive, constantstimulation is desired, these four-pulse bursts are initiated at a givenburst repetition rate correspond ing to theyswitching rate shown in FIG.2A. Such initiation may occur at a rate ranging from [0 to I00 pulsebursts per second to produce a signal of the typedesignated 25 in FIG.2A.

The repetition rate of the pulse burst has been found to affect thequality of the sensation produced. At low frequencies 10 to 20 pulsebursts per second), the sensation has a coarsely rasping quality whichis unpleasant to some subjects. Above 20 pulse bursts per second, thesensation becomes smooth, approximating steady touch rather thanvibration. At higher frequencies (50 pulse bursts per second and above)insufficient time is available for skin recovery in some subjects, andconstant stimulation may tend to fade.

The actual sensation which is achieved varies in intensity as a functionof the current in-milliampercs passed in the steady state or finalcurrent in a single pulse as shown in FlG. 2B. In this respect,application of a rectangular voltage pulse to the skin tends to producea current pulse (FlG. 23) having an initial relatively high spike (theskin charging current), which falls to a steady state conditionoperative to produce the desired tactile sensation. The transientcharging current may be, for example, 20 ma. falling to a steady statevalue of 3 to 6 ma. with a time constant of about 2 microseconds. It hasbeen found that stimuli of less than 2 ma. (steady state) are notperceived by most subjects. On the other hand, stimuli of more than 20ma. (steady state) reliably produce pain. Stimuli of 5 to l0 ma. (steadystate) are optimal with a skin electrode having dimensions of the typesdescribed previously. The initial sensation may tend to be ofthe sharppunctuate type; but after 5 minutes of experience, subjects havereported that the sensation is comfortable, and totally unlike anelectric shock.

FIG. 2C summarizes the foregoing informationv This FIG. show thevariation of, threshold of feeling and threshold of pain (includingpossible variations for different subjects) with electrical stimuluspulses or trains of pulses having the characteristics described above,and depicts the preferred envelope of operation lying between thesethresholds. The abscissa in the FIG. 2C diagram represents the effectivepulse duration, that is the sum of duration of all of the pulses in onetrain. The ordinate is the pulse amplitude in milliamperes. It can beseen from this diagram that the subjective level of sensation can beaccurately predicted from the current, pulse width, and number of pulsesin a train. I

The optimal steady state current level mentioned earlier is maintainedby a constant current source used as a stimulator, by which voltage isautomatically adjusted according to skin resistance to pass a constantcurrent level through each electrode. In the embodiment illustrated inFIG. 1, the hapticon tube 10 itself inherently acts as a constantcurrent source because of the nature of the electron beam current; butif other signal processing arrangements are used, provision for aconstant current should preferably be made. To provide for amplitudeadjustments, a manual control 21 is preferably provided between theoutput of pulse train generator 20 and hapticon tube 10; and thispermits the steady state current level to be adjusted within limits bythe subject in accordance with his own preference and feelings ofcomfort.

A hapticon visual substitution system of the general type shown in FIG.1 may, as mentioned previously, be readily mounted in or on a pair ofspectacles, as shown in FIG. 3. In place of a lens, spectacle frame 30can readily support a hapticon tube 10; and the stimulus wave generator20 can, particularly if fabricated of integrated circuit components, beconveniently contained within one or both of the side temple members 31of the spectacles, e.g., at the location designated 20a. Small,lightweight batteries used to energize generator 20 can be convenientlycarried in the pocket, on the belt, on any convenient location; and canbe connected to the hapticon and associated wave generator by a cable32, a portion of which may also act as a connector the skin stimulatormatrix.

For monocular -vision, only one hapticon need be used, mounted on sideonly of the spectacles; and the other side can remain blank, or containa counterweight dummy unit. For

stereoscopic vision (to be discussed in reference to FIG. 6) a pair ofhapticons can be used. mounted respectively in the two frame openings ofthe spectacles.

Hapticon I is associated with a Fresnel objective lens 33 mounted on anappropriate support 34 extending forward of the spectacle frame 30 toprovide a spacing of 1 inch to I zinches between lens 33 and thephotocathode of hapticon l0. Vaiious types of optics can be used toproduce the best image for a particular application, e.g., zoom,fisheye, wide angle, telephoto, etc.

A zoom optics arrangement employing Fresnel lenses, adapted to bemounted adjacent the position of lens 33 in FIG. 3, is shown in FIG. 3A.Such a system may comprise three Fresnel (or conventional) lenses 33a,33b, and 330 associated with mirrors 35a35d inclusive. Lens 33a, whichis the forwardmost lens of the system, is preferably replaceable. turretmounted, or otherwise selectable to provide three different conditions,i.e., extreme wide angle (approaching 180), medium angle (e.g., about 30field of vision), or narrow angle (e.g., about viewing to permit variedexamination of an object. Lens 33!) and its associated mirrors 35b and35c are mounted for vertical movement, e.g., as designated at 36,relative to fixed position lenses 33a, 33: and fixed position mirrors35a, 35d; and preferably permit a zoom of l or greater for each lensselected at 330 to provide continuous overlap in the field of view. Zoomcan be effected by a manually controllable adjustment knob adapted toeffect movement of the optics along arrow 36. When the lenses are of theFresnel type, the overall optical arrangement can be extremely light inweight and readily mounted on a spectacles frame of the type shown inFIG. 3 without causing discomfort. It will be appreciated, of course,that other optical arrangements may be employed; and

it will further be appreciated that all of these various opticalsystems, including that shown in FIG. 3A, can be employed in conjunctionwith other forms of the hapticon system, e.g., the head-mountedarrangement of FIG. 4, or a hand held unit.

When the hapticon photomultiplier system of the present invention ismounted in a pair of spectacles, or worn on the head or in a hat, normalhead movements produce sufficient image translation across the skin toprevent fading of the image or adaptation to a fixed pattern ofstimulus. thus insuring continued perception of visible objects. In sucharrangements, head movements replace the normal nystagmus movements ofthe eyes which are responsible for maintaining continued perception ofvisual images in sighted individuals. (Images fixed upon the retinaquickly fade from perception in the same manner as fixed tactileimages.)

A head mounted hapticon system of the general type already described inreference to FIG. 1, is shown in FIGS. 4 and 4A. The hapticon itself hasbeen designated 40, and is associated with appropriate optics of thetypes already described and diagrammatically illustrated at 41. Theseelements are, in the manner already described, associated with anappropriate pulse train generator package shown at 42. The severalcomponents 4042 are, in one specific example shown in the drawings,mounted upon a semiflexible backing member 43 provided on itsundersurface with an array of preferably flexible probes 44, e.g., I000such probes, adapted to push through the subjects hair and contact theskin of his scalp on one-eighth inch centers. The elements 43, 44 thusconstitute an electrode blanket in engagement with the scalp of thesubject. The several probes in array 44 can be appropriatelyinterconnected to the output electrodes of hapticon 40 by means ofconductors which extend through and are embedded in backing member 43.

The lens and hapticon portions 41, 40 of the system are, as illustrated,mounted toward the front of the electrode blanket, with the electronicspackage for generating the desired wave train being mounted to the rearof the blanket. However, other packaging arrangements may be employed,e.g., the electronic package 42 may be mounted to the sides of hapticon40 or at some other location. Battery power for short temr operation canbe effected by means of one or more battery units carried 8 by or nearbacking member 43;-but if the batteries employed are relatively heavy orlarge in size. the batteries can be carried in the subjects pocket. onhis belt, or at some other convenient location with a battery cable 45being provided for connection between such a remote battery and theheadmounted hapticon unit.

The system of the present invention readily lends itself to theconveying of color information to the subject. One arrangement of thisgeneral type is shown in FIG. 5A. and comprises three separate hapticonunits 50, SI and 52 each of which is associated with its own opticalsystem 50a, 51a. and 520. Three filters 53, 54 and 55, correspondingrespectively to the three primary colors, register three different colorimages on the hapticon tube 50, SI, and 52 respectively; and the outputelectrodes ofthe hapticons 50-52 are connected in parallel with oneanother and are then connected to stimulus electrodes in engagement withthe subject s skin.

The hapticon units 5I)52 each comprise a hapticon system of the generaltype shown in FIG. I; and each such system includes a pulse traingenerator operative in the manner already described. Each hapticonsystem should operated at a separate distinctive carrier or modulatingfrequency, whereby three distinctly different paralleled outputs areconveyed to the skin representative ofthe three primary colors. Fulloctave frequency differences can be employed for the three modulatingfrequencies, e.g.. modulating frequencies such as 15, 3t), and 60 Hz.respectively may be employed. Altematively. the modulating frequenciesmay be incommensurate multiples of one another, e.g., I5, 27, and 53 Hz.By training and practice the subject becomes adept at distinguishing thedifferent frequencies of stimulation from one another, and becomes ableto correlate the different frequencies sensed with different colors.Thus the single set of stimulating electrodes is operative to conveycolor information to the skin of a blind subject.

An alternative arrangement is shown diagrammatically in FIG. 5B. In thearrangement of FIG. 5A the three hapticon units 50-52 employ threedifferent envelopes for the respective hapticon tubes; but this is notmandatory. A singleevacuated envelope 56 can be provided with threespaced microchanncl plates 18a, 18b, and 180, each of which isassociated in turn with its own separate pulse train generator. Thephotocathode on the front face of this single envelope may be dividedinto three segments 56a, 56b and 56c which are covered respectively bythree primary color filters, and as sociated respectively with threelens systems operative to image objects on each photocathode segment.The output electrodes of this composite hapticon tube would accordinglyprovide the desired color information in the same manner al readydescribed in reference to FIG. 5A.

The single-envelope hapticon arrangement of FIG. 58 need not employthree separate microchannel plates, but can actually employ a singlemicrochanncl plate of sufficient size to cooperate with the threeseparate photocathodc areas 56a, 56b, and 56c, with three separatewaveform generators connected respectively between the commonmicrochannel plate and the three photocathode segments. In analternative arrangement of this type, different groups of tubes in thecommon microchannel plate cooperate respectively with the differentphotocathode areas to achieve the desired color informationtransmission.

A further arrangement is shown in FIG. 5C, and embodies a singlehapticon 57 associated with lens system 58 and with a rotating colorfilter disc 59 interposed therebetween. A 1 inch diameter hapticon 57requires only a 2 inch diameter rotating color filter disc. Severalmodulating frequencies should again be applied to the hapticon, and thefrequency used at any given time should be switched in synchrony withrotation of disc 59 to effect a desired modulation corresponding to theprimary color filter segment then in front of hapticon 57. Red huescould be represented eg by a 20 Hz. modulating frequency, green by a 30Hz. modulating frequency, and blue -'by-a 40 Hz; modulating frequency;but any alternative frequencies, such as those suggested earlier inreference to FIG. A, could also be employed. The color informationoutput achieved by the system of HG. 5C takes the form ofserialinformation, rather than parallel information.

To achieve a stereoscopic presentation, a single hapticon tube connectedto an electrode matrix for stimulating the skin may be caused to shiftits point of view alternately between left and right positions whosebase line separation corresponds approximately to the interocularseparation. (Enhanced depth perception may be achieved bywiderseparation.) An arran'gement of this type is shown in FIG. 6 whereinhapticon so has its output couple via cable 61 to an electrode matrix62.Visual information is impressed upon hapticon 60 by means of a pair offixed position mirrors 63, 64 cooperating with a rotating or nutatingmirror 65 and an appropriate lens system 66. Mirrors 63 and 64 arelooked at alternatively by mirror 65 as it shifts between the full lineposition shown in FIG. 6 (for left viewing) and the broken line position650 (for "right view ing).

Rotation or nutation of mirror 65 can be effected by an appropriatedrive mechanism 67, e.g., a small electric motor as sociated with anappropriate gear train or cam arrangement. Operation of drive means 67may also be caused to operate a right-left stimulus generator 68operative to produce a signal on either line 69 or 69a for applicationrespectively to a pair of electrodes 70 and 70a located respectively atthe right hand and left hand edges of electrode matrix 62. The operationis such that, when drive means 67 causes mirror 65 to reflect a righthand point of view image tohapticon 60, generator 68 applies a clueingstimulus to electrode 70 to so inform the subject; and on alternate halfcycles, when mirror 65 applies a left hand view point image to hapticon60, electrode 700 is similarly energized to so inform the subject. Bythis arrangement, therefore, visual information taken from two differentview points is regularly and alternately used to provide energization toelectrode matrix 62; and the subject is simultaneously informed justwhich point of view is being represented at any particular time.

Alternatively, two hapticon tubes can be employed with an electronicswitch providing time sharing between them by connecting the pulsedaccelerating voltage source to each tube in succession.

in the system of FIG. 6, only one plane of an object being viewed isrepresented. This plane is determined by the inter section of the axesof the right and left hand mirrors 63, and 64. By converging the anglesof mirrors 63, 64, the subject can effectively fix, fuse or focus" onnear objects (within feet), making them appear as a single image in thestimulator matrix presentation. Thus, by knowing the mirror angles, asubject can estimate range to a given object. Objects in front of orbehind this focal plane" would appear as double images; and the doubleimage separation would increase with increasing range. Means canaccordingly be provided to adjust the angles of mirrors 63, 64; and theactual mirror angle convergence can be readily calibrated in terms ofrange from the subject.

Due to the fact that the hapticon of the present invention can be madein very small sizes, it is also possible to incorporate a visualsubstitution system of the type described in or on a pair of spectaclesor other appropriate head harness in a manner which makes use of ocularfeedback. An arrangement of this type is shown in FIGS. 7 and 7A. Theblind subject can be provided with mirrored contact lenses for his blindeyes, in an arrangement taking the general fonn shown in FIG. 7A. Thelens 75 shown in FIG. 7A is of the scleral contact type, and carries aplane or curved mirror 76 adapted to be carried by and moved with normaleye movements of the wearer, and cooperating with a spectacles mountedmirror 78 in the manner illustrated in FIGS. 7 and 7A. A scleral lens 75is preferred to the corneal type since it minimizes positionalinstability of the mirror while pennitting movement of the mirror withnormal eye movement; but similar results can be achieved by employing acounterweighted corneal contact lens for supporting a mirror such as 76.

The contact lens arrangement of FIG. 7A cooperates with a pair ofspectacles '77, the side temple pieces of which carry an angled relaymirror 78. lens means 7) and a hapticon 80. Visual information reflectedby contact mirror 76 is relayed by mirror 78 and optics 79 to hapticon80. and may then be coupled as electrical stimulating signals to anelectrode matrix array ofone ofthc types already described. Normal eyemovements operate to direct and change the angle of view of the hapticon80 (or pair of such haptico'ns mounted respectively in the opposing sidepieces of the spectacles 77). By the system of FIG. 7, the normalvestibular and ocular reflexes of the subject are used as part of thevisual substitution system.

While we have thus described preferred embodiments ofthe presentinvention, many variations will be suggested to those skilled in theart, and some of these. variations have already been mentioned. Othervariations will be apparent; and it must therefore be understood thattheforegoing description is intended to be illustrative only and notlimitative of our inven tion. All such variations and modifications asare in accord with the principles described, are meant to fall withinthe scope of the appended claims. I

We claim: v

I. A visual substitution system comprising signal generator meansoperative to produce output signals in the form of a plurality of spacedpulse bursts, each of said bursts comprising a plurality of pulseshaving a repetition rate up to l(),0()0 Hz. and said bursts being spacedfrom one another to occur at a repetition rate up to I00 pulse burstsper second, an artificial receptor responsive to visible information andcoupled to said signal generator means for modulating the amplitude ofsaid pulses in accordance with the brightness of different portions ofsaid visual information. and means for coupling said modulatcd pulses tothe skin of a subject to achieve a patterned electrical stimulation ofthe skin.

2. The system of claim I wherein each of said pulses is sub stantiallyrectangular in waveform, each pulse having a width of between l and 500microseconds.

3. The system of claim 1 wherein said artificial receptor comprises aphotocathode, an array of collector anodes spaced from saidphotocathode. and an image intensifier positioned between saidphotocathode and said anode array, said signal generator means beingelectrically coupled to said image intensifier.

4. The system of claim 3 wherein said image intensifier comprises asecondary emission microchanncl plate.

5. The system of claim 4 wherein the number of anodes in said array isless than the number of channels in said' microchanncl plate, wherebyelectron image beam current from a plurality of said channels isdirected onto each of said anodes.

6. The system of claim 5 including a matrix of spaced electrodes forengagement with the skin of a subject, and conductor means coupling eachof the anodes in said array to a corresponding one of the electrodes insaid matrix.

7. A visual substitution system comprising an artificial receptor forconverting visual information into electrical signals; said receptorcomprising a photocathode, an array of collector anodes spaced from saidphotocathode, and a secondary emission microchanncl plate imageintensifier positioned between said photocathode and said anode array;optical means for projecting visual information onto said photocathode;a matrix of spaced electrodes for engagement with the skin of a subject;and connector means electrically coupling the anodes in said array tothe electrodes in said matrix.

8. The system of claim 7 including pulse generator means coupled to saidartificial receptor for converting said visual information into pulsesof electrical information at said collector anodes.

9. The system of claim 8 wherein said pulse generator means is operativeto convert said visual information into substantially rectangular pulsesoccurring at a repetition rate up to l0,000 Hz.

10. The system of claim 7 wherein said artificial receptor is mounted ona pair of spectacles.

II. The system olclaim 8 wherein said artificial receptor is mounted onthe frontal piece ofa pair of spectacles. said pulse generator meansbeing carried by at least one of the side temple members of said pair ofspectacles.

12. The system of claim 7 wherein said matrix of spaced electrodes ismounted for support on the head of a subject with said electrodes inengagement with the subject's scalp.

13. The system of claim 12 wherein said artificial receptor and saidmatrix of spaced electrodes are mounted on a common support member.

14. The system of claim 7 wherein said photocathode comprises aplurality of spaced photocathode sections, different color filter meanspositioned respectively between said optical means and said spacedphotocathode sections, and a plurality of different frequency modulatingpulse generators coupled respectively to said plurality of photocathodesections.

15. The system of claim 14 wherein said artificial receptor includes aplurality of said microchanncl plates disposed adjacent said pluralityof photocathode sections respectively.

16. The system of claim 15 wherein said plurality of photocathodesections and said plurality of microchanncl plates are mounted in acommon evacuated envelope.

17. The system of claim 15 wherein said plurality of photocathodesections and said plurality of microchanncl plates are mountedrespectively in separate evacuated envclopes.

18. The system of claim 7 wherein said optical means includes at leastone movable element for varying the nature of the visual informationwhich is projected onto said photocathode.

l). The system of claim l8 wherein said movable element comprises amovable mirror for changing the view point of visual informationprojected onto said photocathode, and drive means for regularly varyingthe position ofsaid movable mirror.

20. The system of claim 19 including signal means operative insynchronism with said drive means for coupling signals to the skin otasubject informing the subject of the viewpoint of visual informationbeing projected onto said photocathode 21. The system of claim 18wherein said movable element comprises mirror means mounted for supporton and move ment with the eye of a subject for controlling the directionof view of said artificial receptor.

22. The system of claim 7 comprising a plurality of said artificialreceptors, and control means for rendering different ones of saidreceptors operative at different times.

1. A visual substitution system comprising signal generator meansoperative to produce output signals in the form of a plurality of spacedpulse bursts, each of said bursts comprising a plurality of pulseshaving a repetition rate up to 10,000 Hz. and said bursts being spacedfrom one another to occur at a repetition rate up to 100 pulse burstsper second, an artificial receptor responsive to visible information andcoupled to said signal generator means for modulating the amplitude ofsaid pulses in accordance with the brightness of different portions ofsaid visual information, and means for coupling said modulated pulses tothe skin of a subject to achieve a patterned electrical stimulation ofthe skin.
 2. The system of claim 1 wherein each of said pulses issubstantially rectangular in waveform, each pulse having a width ofbetween 1 and 500 microseconds.
 3. The system of claim 1 wherein saidartificial receptor comprises a photocathode, an array of collectoranodes spaced from said photocathode, and an image intensifierpositioned between said photocathode and said anode array, said signalgenerator means being electrically coupled to said image intensifier. 4.The system of claim 3 wherein said image intensifier comprises asecondary emission microchannel plate.
 5. The system of claim 4 whereinthe number of anodes in said array is less than the number of channelsin said microchannel plate, whereby electron image beam current from aplurality of said channels is directed onto each of said anodes.
 6. Thesystem of claim 5 including a matrix of spaced electrodes for engagementwith the skin of a subject, and conductor means coupling each of theanodes in said array to a corresponding one of the electrodes in saidmatrix.
 7. A visual substitution system comprising an artificialreceptor for converting visual information into electrical signals; saidreceptor comprising a photocathode, an array of collector anodes spacedfrom said photocathode, and a secondary emission microchannel plateimage intensifier positioned between said photocathode and said anodearray; optical means for projecting visuAl information onto saidphotocathode; a matrix of spaced electrodes for engagement with the skinof a subject; and connector means electrically coupling the anodes insaid array to the electrodes in said matrix.
 8. The system of claim 7including pulse generator means coupled to said artificial receptor forconverting said visual information into pulses of electrical informationat said collector anodes.
 9. The system of claim 8 wherein said pulsegenerator means is operative to convert said visual information intosubstantially rectangular pulses occurring at a repetition rate up to10,000 Hz.
 10. The system of claim 7 wherein said artificial receptor ismounted on a pair of spectacles.
 11. The system of claim 8 wherein saidartificial receptor is mounted on the frontal piece of a pair ofspectacles, said pulse generator means being carried by at least one ofthe side temple members of said pair of spectacles.
 12. The system ofclaim 7 wherein said matrix of spaced electrodes is mounted for supporton the head of a subject with said electrodes in engagement with thesubject''s scalp.
 13. The system of claim 12 wherein said artificialreceptor and said matrix of spaced electrodes are mounted on a commonsupport member.
 14. The system of claim 7 wherein said photocathodecomprises a plurality of spaced photocathode sections, different colorfilter means positioned respectively between said optical means and saidspaced photocathode sections, and a plurality of different frequencymodulating pulse generators coupled respectively to said plurality ofphotocathode sections.
 15. The system of claim 14 wherein saidartificial receptor includes a plurality of said microchannel platesdisposed adjacent said plurality of photocathode sections respectively.16. The system of claim 15 wherein said plurality of photocathodesections and said plurality of microchannel plates are mounted in acommon evacuated envelope.
 17. The system of claim 15 wherein saidplurality of photocathode sections and said plurality of microchannelplates are mounted respectively in separate evacuated envelopes.
 18. Thesystem of claim 7 wherein said optical means includes at least onemovable element for varying the nature of the visual information whichis projected onto said photocathode.
 19. The system of claim 18 whereinsaid movable element comprises a movable mirror for changing the viewpoint of visual information projected onto said photocathode, and drivemeans for regularly varying the position of said movable mirror.
 20. Thesystem of claim 19 including signal means operative in synchronism withsaid drive means for coupling signals to the skin of a subject informingthe subject of the viewpoint of visual information being projected ontosaid photocathode.
 21. The system of claim 18 wherein said movableelement comprises mirror means mounted for support on and movement withthe eye of a subject for controlling the direction of view of saidartificial receptor.
 22. The system of claim 7 comprising a plurality ofsaid artificial receptors, and control means for rendering differentones of said receptors operative at different times.